According to Kathleen A. Jackson, INTRUSION DETECTION SYSTEM (IDS) PRODUCT SURVEY, Version 2.1, Los Alamos National Laboratory 1999, Publication No. LA-UR-99-3883, Chapter 1.2, IDS OVERVIEW, intrusion detection systems attempt to detect computer misuse. Misuse is the performance of an action that is not desired by the system owner; one that does not conform to the system's acceptable use and/or security policy. Typically, misuse takes advantage of vulnerabilities attributed to system misconfiguration, poorly engineered software, user neglect or abuse of privileges and to basic design flaws in protocols and operating systems.
Intrusion detection systems analyze activities of internal and/or external users for explicitly forbidden and anomalous behavior. They are based on the assumption that misuse can be detected by monitoring and analyzing network traffic, system audit records, system configuration files or other data sources (see also Dorothy E. Denning, IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-13, NO. 2, February 1987, pages 222-232).
The types of methods an intrusion detection system can use to detect misuse can vary. Essentially, there are two main intrusion detection methods known, which are described for example in EP 0 985 995 A1 and U.S. Pat. No. 5,278,901.
The first method uses knowledge accumulated about attacks and looks for evidence of their exploitation. This method, which on a basic level can be compared to virus checking methods, is referred to as knowledge-based, also known as signature-based or pattern-oriented or misuse detection. A knowledge-based intrusion detection system therefore looks for patterns of attacks while monitoring a given data source. As a consequence, attacks for which signatures or patterns are not stored, will not be detected.
According to the second method a reference model is built, that represents the normal behavior or profile of the system being monitored and looks for anomalous behavior, i.e. for deviations from the previously established reference model. Reference models can be built in various ways. For example in S. Forrest, S. A. Hofineyr, A. Somayaji and T. A. Longstaff; A Sense of Self for Unix Processes, Proceedings of the 1996 IEEE Symposium on Research in Security and Privacy, IEEE Computer Society Press 1996, pages 120-128, normal process behavior is modeled by means of short sequences of system calls.
The second method is therefore referred to as behavior-based, also known as profile-based or anomaly-based. Behavior-based intrusion detection, which relies on the assumption that the “behavior” of a system will change in the event that an attack is carried out, therefore allows to detect previously unknown attacks, as long as they deviate from the previously established model of normal behavior. Under the condition that the normal behavior of the monitored system does not change, a behavior-based intrusion detection system will remain up-to-date, without having to collect signatures of new attacks.
However, since the behavior of a system normally changes over time, e.g. due to changes in the activities of authorized users or installation of new or updated system elements, without immediate adaptation of the used reference model deviations from the modeled behavior will frequently be detected without any intrusions taking place. Behavior-based intrusion detection systems will therefore normally produce a large number of false alarms (false positives) deriving from non-threatening events.
Knowledge-based intrusion detection systems tend to generate fewer false alarms. However, depending on the quality of the stored knowledge of known attacks and the condition of the monitored system these systems may also produce numerous false alarms which can not easily be handled by human system administrators. For example, some network applications and operating systems may cause numerous ICMP (Internet Control Message Protocol) messages (see Douglas E. Corner, INTERNETWORKING with TCP/IP, PRINCIPLES, PROTOCOLS, AND ARCHITECTURES, 4th EDITION, Prentice Hall 2000, pages 129-144), which a knowledge-based detection system may interpret as an attempt by an attacker to map out a network segment. ICMP-messages not corresponding to normal system behavior may also occur during periods of increased network traffic with local congestions.
It is further known that an intrusion detection system may interpret sniffed data differently than the monitored network elements, see Thomas H. Ptacek, Timothy N. Newsham, Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection, Secure Network Inc., January 1998, which under certain conditions could also lead to false alarms.
False alarms, appearing in large numbers, are a severe problem because investigating them requires time and energy. If the load of false alarms in a system gets high, human system administrators or security personnel might become negligent. In Klaus Julisch, Dealing with False Positives in Intrusion Detection, RAID, 3rd Workshop on Recent Advances in Intrusion Detection, 2000, it is described that filters could be applied in order to remove false alarms. Filters can also use a knowledge-based approach (discarding what are known to be false positives) or a behavior-based approach (discarding what follows a model of normal alarm behavior). Either way, maintaining and updating models or knowledge bases of filters and intrusion detection systems requires further efforts.
It would therefore be desirable to create an improved method and a system for processing alarms triggered by a monitoring system such as an intrusion detection system, a firewall or a network management system in order to efficiently extract relevant information about the state of the monitored system or activities of its users.
It would further be desirable for this method and system to operate in the presence of a large amount of false alarms, which are received at a rate that can not be handled efficiently by human system administrators.
Still further, it would be desirable to receive the results of said data processing procedures, in a short form but with a high quality of information, that can easily be interpreted by human system administrators or automated post processing modules.